A computing system typically stores data in logical containers, e.g., files, directories, volumes, logical units, as blocks of data. An operating system can store data blocks on a disk or other storage device in any convenient size, for example, a block size could be set to 4 kilobytes (KB), such as is convenient for transfer of blocks to random access memory which often uses a 4 KB memory page size. However, any convenient size either larger or smaller could be used. Where a logical container is 1 megabyte (MB) and a block size is 4 KB, for example, the logical container is stored as 256 blocks.
Further, logical containers often include more than just user data. Operating systems also store meta-data such as attributes and extended attributes. An attribute is a unit of meta-data associated with a logical container defining a property of a logical container. Attributes can refer to, for example last modified date & time, creation date, and read/write access permission. An extended attribute is a unit of meta-data including additional information about a logical container. Extended attributes can vary between operating systems. Examples of extended attributes can include access control lists (ACL), streams, and object IDs. Depending on the operating system used, the location, types and quantity of meta-data can vary.
Because stored data can become lost or corrupted for various reasons, it is desirable to make backup copies of logical containers stored in a computing system. It is also sometimes desirable to backup associated meta-data. One way of backing up data or meta-data is to perform a disk-to-disk backup. A disk-to-disk backup involves copying logical containers from a first disk to a second disk, typically over a network, but sometimes by direct connection; all attributes and extended attributes are copied along with the logical containers.
Should the logical containers on the first disk be lost or corrupted, the logical containers can be restored from the second disk. When logical containers are transmitted over a network during a backup, a socket connection is often used. A socket connection allows for inter-process communication such as between an operating system and a backup system. In some implementations, over the socket connection logical containers are transferred one at a time from the first disk to the second disk until all logical containers have been transferred. However, socket connections sometimes fail due to network failure, overloading, or other unforeseen difficulties.
Connection failure can be particularly inconvenient when a large logical container has been only partially transferred. Where network data transmission is expensive, the costs associated with re-transmission can be wasteful. The re-transmission of a large logical container can take a significant amount of time, and where the connection fails repeatedly, the logical container might not ever be completely backed up.
Further, it may be possible to backup part of a logical container and after recovering from a network connection failure, begin to copy the logical container where the backup process left off. However, where a logical container is partially backed up, but its attributes or other meta-data have changed since beginning the backup, continuing the backup process can lead to inconsistencies in the logical container. For example, the backup can include some data that is new and other data that is old. It can become necessary to begin the backup process of the logical container again, resulting in wasted time and computing resources.